The manual processing method is mainly used for the detection, tracking and information editing of key objects in indoor scene video, which has the problems of low efficiency and low precision. In order to solve the problems, a new learning method of indoor scene semantic annotation based on texture information was proposed. Firstly, the optical flow method was used to obtain the motion information between video frames, and the key frame annotation and interframe motion information were used to initialize the annotation of non-key frames. Then, the image texture information constraint of non-key frames and its initialized annotation were used to construct an energy equation. Finally, the graph-cuts method was used for optimizing to obtain the solution of the energy equation, which was the non-key frame semantic annotation. The experimental results of the annotation accuracy and visual effects show that, compared with the motion estimation method and the model-based learning method, the proposed learning method of indoor scene semantic annotation based on texture information has the better effect. The proposed method can provide the reference for low-latency decision-making systems such as service robots, smart home and emergency response.